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Abstract 

We investigate the spatial distribution and the global frequency of agents who can either 
cooperate or defect. The agent interaction is described by a deterministic, non-iterated prisoner's 
dilemma game, further each agent only locally interacts with his neighbors. Based on a detailed 
analysis of the local payoff structures we derive critical conditions for the invasion or the spatial 
coexistence of cooperators and defectors. These results are concluded in a phase diagram that 
allows to identify five regimes, each characterized by a distinct spatiotemporal dynamics and 
a corresponding final spatial structure. In addition to the complete invasion of defectors, we 
find coexistence regimes with either a majority of cooperators in large spatial domains, or 
a minority of cooperators organized in small non-stationary domains or in small clusters. The 
analysis further allowed a verification of computer simulation results by Nowak and May (1993). 
Eventually, we present simulation results of a true 5-person game on a lattice. This modification 
leads to non-uniform spatial interactions that may even enhance the effect of cooperation. 
Keywords: Prisoner's dilemma; cooperation; spatial 5-person game 



1 Introduction 



The evolution of cooperation has been extensively studied in a biological, social and ecological 
context |, |, |, 0, [yj. |l|, H, U, H HI M> B 0- In order to obtain a general theory for 
this, in particular the so called Prisoner's Dilemma (PD) (see also Sect. |2.2| ) - an evolutionary 
game introduced by Rapoport and Chammah (see also [|3gf| ) - has been widely investigated 
|0, ||, 13, 14, 17, |(], 26, ^8, 30, 34, 43]. Based on PD investigations, in his seminal work Axelrod 
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H has shown that cooperation can emerge as a norm in a society comprised of individuals with 
selfish motives - for a review of this development in the last twenty years see e.g. 

The Prisoner's Dilemma is based on the precondition that it pays off to be not cooperative, i.e. 
to defect in a cooperative environment, this way taking a "free ride" at the costs of those how 
are cooperating, e.g. for a commong good. This type of problem has been also recognized as the 
"tragedy of the commons" [18]. Many investigations of the Prisoner's Dilemma consider a so-called 



iterated game ]2|, ||, ||, 14, 21, 27, |35|| , where the players interact consecutively a given number 
of times, n g > 2. It makes sense only if the players can remember the previous choices of their 
opponents, i.e. if they have a memory of n m < n g — 1 steps. Then, they may be able to develop 
different strategies - such as the famous "tit for tat" || || - based on their past experiences with 
their opponents. 

In this paper, we are only interested in a non-iterated PD game, n g = 1, which is also called a 
"one-shot" game. In this case the players - or the agents as we call them here - do not develop 
strategies, they can rather choose between two different actions, to cooperate (C) and to defect (D). 
It can be shown (see also Sect. 2^) that in the one-shot game defection is the only evolutionary 
stable strategy (ESS) if each player interacts with any other player. Such a population is 
called panmictic, and their dynamics can be predicted within a mean-field analysis. Interestingly, 
this picture changes if a spatial structure of the population is considered, i.e. if the interaction 
between agents is locally restricted to their neighbors [17, [H], ^4], |3l], [52], 35, |4^]. Then, a stable 
spatial coexistence between cooperators and defectors becomes possible under certain conditions 



1 EM 



The current paper focuses on the spatial interaction of cooperators and defectors both analytically 
and by means of computer simulations. We consider an agent population placed on a square lattice 
and simulate the dynamics of the population by means of a cellular automata defined in Sect. 
2.1. The interaction between the agents is modeled as a n-person game, i.e. n agents interact 
simultaneously. In this paper, a 5-person game is considered defined by the spatial structure of 
each agent's four nearest neighbors. It is already known that multi-person games produce a rather 
complex collective dynamics jl], 11] which becomes even more difficult in the spatial case. Therefore, 
in Sects. 3 and 4, we derive analytically critical conditions for the invasion or the coexistence of 
cooperating and defecting agents. In particular, we verify analytically the critical parameters found 
by Nowak and May |3!| by means of computer simulations. A very detailed investigation leads us 
to a phase diagram that allows to distinguish between five different dynamic regimes for the spatial 
evolution. Examples for these are demonstrated in Sect. 4 by means of computer simulations. In 
Sect. 5 we present a modification of the model, that to our knowledge for the first time investigates 
a true 5-person game on the lattice. The results indicate that these modifications may lead to an 
increase of cooperation in the spatial population. 
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2 Model of Cooperation 

2.1 Defining the cellular automaton 

We consider a model of two types of agents, cooperators (C) and defectors (D). The disctinction 
is made by means of a variable 9 £ {0, 1}, where 1 refers to cooperation and refers to defection. 
Whether an individual agent i belongs to the subpopulation of C or D is indicated by a variable, 
6i £ 9. The total number of agents is constant, thus the global frequency fg is defined as: 

N = ^2N e = N + Nx = const. 

e 

fe = ^; / = /i = l-/ (i) 

In the following, the variable / shall refer to the global frequency of the cooperating agents. 

For the spatial distribution of the agents let us consider a two-dimensional cellular automaton (CA) 
(or a two-dimensional square lattice) consisting of N cells. Since the cells can be numbered consec- 
utively, each cell is identified by the index i € N, that also refers to its spatial position. We note 
that in most two-dimensional CA the position of the cells are identified by ij coordinates referring 
to the two dimensions, however in order to define neighborhoods and to simplify the notations, we 
will use only index i for the spatial position. This implies that the description presented below may 
also apply to one-dimensional CA provided the neighborhoods are defined accordingly. 

Each cell shall be occupied by one agent, thus each cell is characterized by a discrete value 9i £ {0, 1} 
indicating whether it is occupied by a cooperator or a defector. The spatiotemporal distribution of 
the agents is then described by 

® = {e 1 ,9 2 ,...,e N ) (2) 

Note that the state space Q of all possible configurations is of the order 2^. 

For the spatial evolution of described in the following, we have to consider the occupation distri- 
bution of the local neighborhood that surrounds each cell i. Let us define the size of a neighborhood 
by n (that also includes cell i), then the different neighbors of i are characterized by a second 
index j = 1, n — 1, where the numbering starts with the nearest neighbors (cf. Fig. ||). Note that 
@i — @io (i- e j = 0)- The number of nearest neighbors of cell i shall be denoted as m, whereas the 
number of only the second nearest neighbors is denoted as r. Obviously, n = m + r + 1. For further 
use, we define the local occupation 9^ of the nearest neighborhood (without cell i) as: 



(3) 
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Figure 1: Sketch of the two-dimensional cellular automaton. It shows a neighborhood of size n of 
cell i where the neighbors are labeled by a second index j = 1, ...,n — 1. The value Qy £ {0, 1} 
indicates whether the cell is occupied either by a defector or a cooperator. The m nearest neighbors 
are shown in darker gray, the r second nearest neighbors in lighter gray. 

2.2 Rules of the Game 

In order to specify the interaction between the agents, we adopt a well known prototype from game 
theory. It assumes that each agent i has two options to act in a given situation, to cooperate (C), 
8i = 1, or to defect (D), Oi = 0. Playing with agent j, the outcome of this interaction depends on 
the action chosen by agent i, i.e. C or D, without knowing the action chosen by the other agent 
participating in a particular game. It is described by a payoff matrix, which for the 2-person game 
has the following form: 
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Suppose, agent i has chosen to cooperate, then its payoff is R if the other agent j has also chosen to 
cooperate (without knowing about the decision of agent i), but S if agent j defects. On the other 
hand, if agent i has chosen not to cooperate, then it will receive the payoff T if agent j cooperates, 
but P if agent j defects, too. 
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In a special class of games, the so-called Prisoner's Dilemma (PD), the payoffs have to fulfill the 
following two inequalities: 

T> R> P> S (5) 
2R>S + T (6) 

The known standard values are T = 5, R = 3, P = 1, S = 0. This means in a cooperating 
environment, a defector will get the highest payoff. From this, the abbreviations for the different 
payoffs become clear: T means (T)emptation payoff for defecting in a cooperative environment, S 
means (S)ucker' payoff for cooperating in a defecting environment, R means (R)eward payoff for 
cooperating in a likewise environment, and P means (P)unishment payoff for defecting in a likewise 
environment. 

In any one round (or "one-shot") game, choosing action D is unbeatable, because it rewards the 
higher payoff for agent i whether the opponent chooses C or D. At the same time, the payoff for 
both agents i and j is maximized when both cooperate. 

A simple analysis shows that defection is a so-called evolutionary stable strategy (ESS) in a one- 
shot PD. If any two agents are able to interact and the number of cooperators and defectors in 
the population is given by N\ and Nq = N — N\ respectively, then the expected average payoff for 
cooperators will be a\ = (R x Ai + S x An)/A. Similarly the expected average payoff for defectors 
will be an = (T x Ai + P x Nq)/N. Since T > R and P > S, an is always larger than a± for a 
given number Ai, and pure defection would be optimal in a one-shot game. Even one defector is 
sufficient to invade the complete population of A — 1 cooperators. 

This conclusion holds for a so called panmictic population where each agent interacts with all other 
A — 1 agents. But in this paper, we are mainly interested in the spatial effects of the PD game. 
Therefore, let us assume that each agent i interacts only with the m agents of its neighborhood, 
eq. (§). In evolutionary game theory this is called a n-person game, where n = m-(-l = 5in the 
given case. Each game is played between n players simultaneously, and the payoff for each player 
depends on the number of cooperators in its group, s < n. Then the payoff matrix, eq. (||) has to 
be extended according to the number of players, but it still has to fullfill the following conditions 
according to the Prisoner's Dilemma rationale [46|: 



1. Given a fixed number s of cooperators in the group, defection pays always more than coop- 
eration: 

a S Q>a{; s = 0, 1,2, ...,n - 1 (7) 

where is the payoff for each defector in the group of size n and a\ is the payoff for each 
cooperator. 

2. The payoff increases for both cooperators and defectors, as number s of cooperators in the 
group increases: 

a? > a\~ l ; a s > a^ 1 ; s = 1, 2, n - 1 (8) 
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3. The average payoff of the group increases with an increasing number of cooperators: 

a{s + a s (n - s) > a* _1 (s - 1) + ag -1 (n - s + 1) (9) 

There are various ways to fulfill the conditions (|^)-(|9P- One simple method uses the concept of the 
2-person game described above to calculate the payoff of an agent in the n-person game from the 
accumulated payoffs of (n — 1) 2-person games. I.e. for a neighborhood of 5, the 5-person game 
is decomposed into 2-person games, that may occur independently, but simultaneously p3fl . This 
method will be also used in the current paper. 

With respect to the spatial interaction in the neighborhood, we will further assume that each 
agent interacts only with the m nearest neighbors |32|. In Sect. [|, we will also shortly discuss 
the case where each agent in the neighborhood of size n interacts with every other agent in this 
neighborhood (which sometimes also involves interaction with second nearest neighbors, as Fig. |l 
indicates). These assuptions will allow us to further use the payoff matrix of the 2-person game, 
eq. (0j), and to reduce the interaction in the neighborhood of size n to n — 1 simultaneous 2-person 
games played by agent i. 

In order to introduce a time scale, we define a generation G to be the time in which each agent has 
interacted with its m nearest neighbors a given number of times, denotes as n g . In our case n g = 1 
(which is called a one-shot game), thus the total number of interaction during each generation is 
roughly N m/2. We note here that n g > 1 can play an important role for the invasion of cooperation 
into a heterogeneous population, || [L4|] . 

We assume that during each generation the state Oi of an agent is not changed while it interacts 
with its neighbors simultaneously. But after a generation is completed, 6i can be changed based on 
the following considerations: From the interaction with its neighbors, each agent i receives a payoff 
ai that depends both on its current state 6i, i.e. whether agent i has cooperated or not, and on 
the 9i ■ of its neighbors. The fraction of cooperators and defectors in the neighborhood of agent i is 
given by: 

^ m 

4 = -E%,; zt e) = i-4 (fie {o,i}) (io) 

Tit •* 

3=1 

where 5 xy means the Kronecker delta, which is 1 only for x = y and zero otherwise. According to 
the payoff matrix of eq. @, the payoff of agent i is then defined as: 

Oi(0i) = 6i 9i \z\ R + z° s] + 5 m k 1 T + z? P] (11) 



Note that the additive calculation of the payoff according to eqs. (|10|) , flll|) is based on the assumtion 
of the m simultaneous, but independent 2-person games played by agent i. 

The payoff cij is then compared to the payoffs of all neighboring agents, in order to find the 
maximum payoff within the local neighborhood during that generation, max{aj,Oi }. If agent i 
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has received the highest payoff, then it will keep its Oi, i.e. it will continue to either cooperate or 
to defect. But if one of its neighbors j has received the higher payoff, then agent % will adopt the 
behavior of the respective agent. This also implies that it may keep its previous behavior if the 
more successful agent has the same 6j. If 

j* = arg maxj =0 ,...,m a ij (12) 

defines the position of the agent that received the highest payoff in the neighborhood, the update 
rule of the game can be concluded as follows: 



i {G + l) = e i JG) (13) 



which means specifically: 



We note that the evolution of the system described by eq. ( |i~4| ) is completely deterministic, results 



for stochastic CA have been discussed in [12 



2.3 First insights from the CA simulations 

Eventually, to give an impression of the spatial dynamics that arises from the game desribed above, 
we have conducted a computer simulation of the CA, using periodic boundary conditions and a 
simultaneous update. The global frequency of cooperators was initially set to /(0) = 0.5. The 
results are shown in the time series of Fig. [2] and the time dependence of f(G) in Fig. [31 

From the computer simulations, we notice the following interesting features of the spatial game: 
Starting with an initial random distribution of cooperators and defectors, we observe the formation 
of spatial domains dominated by either cooperators or defectors. In the very early stages of evo- 
lution, the cooperators concentrate in only a few small clusters that are like islands in the sea of 
defectors. But then these cooperating clusters increase in size, i.e. the cooperators invade into the 
domains of defectors, until in the long run they are the global majority. At all times, we observe a 
spatial coexistence between cooperators and defectors in different domains, instead of a complete 
spatial separation into two domains. We further notice that in the long run the spatial pattern 
becomes stationary (with very small periodic changes that can be also noticed in Fig. ||). 

In order to generalize the above results that have been obtained from only a single run of the CA, 
we need to answer the following questions: 

• Under what conditions will there be a spatial coexistence of cooperators and defectors, and 
when will we observe the extinction of either cooperators or defectors? 
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Figure 2: Time series of the spatial distribution of cooperators (grey) and defectors (black) on a CA 
of size TV = 40 x 40. The time is given by the numbers of generations in brackets. Initial condition: 
/(0) = 0.5, random spatial distribution of cooperators and defectors. Parameters for the payoff 
matrix, eq. (|): {R; S; T; P} = {3.0; 0.0; 3.5; 0.5} (region A, see Sect. fO). 
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Figure 3: Global frequency of cooperators, /, eq. ([[]), vs. time (number of generations). The data 
have been obtained from the time series of Fig. 

• What does the variety and the size of the spatial domains depend on? 

• Will the dynamics always lead to stationary patterns, or will there be also non-stationary 
patterns in the long run? 

• How does the dynamics changes if a larger local neighborhood is considered in the update 
rule (i.e. if the second nearest neighbors are included, too)? 

These questions will be investigated in detail in the following sections. In particular, we will derive 
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analytical results for the conditions that need to be fulfilled by the payoff matrix, eq. 
to find certain spatial patterns. 



in order 



3 Invasion vs. coexistence of cooperating and defecting agents 
3.1 Payoffs for local configurations 

As pointed out in the previous section, the change of agent i's "behavior", 0j — > (1 — Of), depends 
on the payoff of the agents in the immediate neighborhood, which in turn depends on the local 
occupation, eq. (||). In order to derive analytical conditions for the transition of 0j, we thus 
have to look more closely at the possible local occupation patters that shall in the following be 
described by a term Kg. Here, 9 € {0, 1} describes whether the center cell is occupied by either a 
defector or a cooperator, and s € {0, 1, 2, 3, 4} gives the total number of cooperators in the nearest 
neighborhood. Examples for possible configurations in the neighborhood of a defector are shown in 
eq. (ED: 

D 
D 

C 



C 
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c 



c c 
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c 



D C 



D D 



D 
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C 



D D 



D 
D 
D 



D 



(15) 



*3 



K 2 
^0 



Of course, for any given number s S {1,2,3}, there are different local occupation patterns possible 
that may result from exchanging the positions of the agents in the neighborhood. But, as we have 



seen from eq. (11), it is not really the position of agents that matters here, but only the number 
of cooperators and defectors in the local neighborhood, eq. (|i~0|). Therefore, Kq provides sufficient 
information about the local configuration. The possible configurations for Kf, where a cooperator 
occupies the center cell are analogous to eq. (p~5|) . 

For each possible realization of Kn, the respective payoff of the agent in the center, a s B can be 



calculated according to eq. (11). The results are given by eq. ( jig) for cooperators and defectors 
both in terms of an analytical expression and for a particular realization of the payoff matrix, eq. 
(|), {R;S;T;P} = {3; 0; 5; 1} (which are the "classical" values): 



cooperator 



defector 



af = 


R 
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T 


= 5.0 


af = 
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(16) 



9/133 



Frank Schweitzer, Laxmidhar Behera, Heinz Miihlenbein: 
Evolution of Cooperation in a Spatial Prisoner's Dilemma 
Advances in Complex Systems, vol. 5, no. 2-3, pp. 269-299 



From eq. (16), we notice that the payoff a s e increases with the number of cooperators s in the 
neighborhood, i.e. for a particular 

a'eXi'f 1 ( S £{1,...,4}) (17) 
yields. Further, taking the relations of eqs. @ into account, it is obvious that always 

Oq = max{aQ,a*} 

a? = mm{a s ,al} (18) 
a s > a{ (se {0,1,..., 4}) 

Comparing eqs. (|l7|), ([Tg|) with the conditions we can also verify that the payoffs in the 

current form fulfill the general requiremements for the n-person PD. Based on these general con- 
siderations, in the following we discuss the different conditions that arise in the spatial case for the 
invasion or the coexistence of cooperators and defectors. 



3.2 Conditions for invasion and coexistence 

Basically, the update rule of eq. (|l~4T ) determines whether a cell is "invaded" by a particular sub- 
population (i.e. whether the agent adopts the respective behavior). It depends on the local payoff 
received in comparison with the payoffs of the neighboring sites. In order to find the conditions 
for invasion and coexistence, we need to concentrate on the border region between the domains of 
cooperators and defectors that is sketched in Fig. ||. Kf describes the local configuration inside the 
domain of cooperators, while K® describes the local configuration inside the domain of defectors. 
Further, the possible configurations that can be found only in the border region are given. The 
local configurations K\ and K\ are not listed there because they are completely unstable. These 
configurations receive the lowest payoff and therefore, if they initially exist, vanish within the next 
generation and do not need to be taken into account for the discussion of the further evolution. 

In order to elucidate the dynamics at the border, we will separately discuss the two possible cases: 
(i) invasion of configurations Kq "owned" by defectors into neighboring configurations Kf "owned" 
by cooperators, and (ii) the reverse case. Note that the cooperator (1) and the defector (0) are 
always on adjacent sites. The invasion process, i.e. the occupation of the center cell by an agent of 
the opposite subpopulation shall be indicated by an arrow =>. The results for case (i) are listed in 
Table |] and are explained below. 

The defective behavior can invade the whole spatial population without any exception (in a de- 
terministic case), if the lowest possible payoff for defectors, aj, is larger than the highest possible 
payoff for collaborators, af. Actually, the payoff a[j does not matter here, because in the respec- 
tive configuration K®, there is no cooperator left to adopt the defective strategy. Because always 
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Figure 4: Sketch of the possible neighboring configurations in a border region between domains of 
cooperators (Kf) and defectors (Kq). 

Table 1: Invasion of configurations Kq "owned" by defectors into configurations Kf "owned" by 
cooperators in the border region, Fig. ||[ 



confix 


juration 


necessary conditions 


stationary state 


Kl 




a\ > a* 


complete invasion 


Kl 


Kf 
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> aj 


unstable 


K 2 
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af > ag 


coexistence (d) 


K 3 




Oq > af 




K 3 




Op < af 
aj < a\ 


coexistence (b) 



ag > a\ > Oq > ag, eq. (|T^), this leads to the necessary condition for the complete invasion of 
defectors into the domain of cooperators: 

a\ > af (19) 

The two remaining cases that lead to coexistence will be explained together with the next table, as 
well as the special case of Kq. 

For the case (ii), invasion of configurations Kf "owned" by cooperators into configurations Kq 
"owned" by defectors, the results for the possible cases are listed in Table ^. First, we notice that a 
complete invasion of cooperators into the whole population is not possible, because the necessary 
condition a\ > ag, i.e. the lowest possible payoff for cooperators is larger than the highest possible 
payoff for defectors, is never fulfilled. Further, from eq. (|i"8| ) we realize that configuration Kq cannot 
be invaded by K( because the former payoff is always higher. But configuration K( could always 
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Table 2: Invasion of configurations Kf "owned" by cooperators into configurations Kq "owned" by 
defectors in the border region, Fig. 



confix 
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coexistence (a) 
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Kf 
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not possible 
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Kf 
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invade a neighboring configuration Kq if the condition 

afya^ 1 (se {2,3,4}) (20) 

would be satisfied. As eq. ( |l6| ) shows, this is not always the case, but provided suitable values of 
T and P, it can be possible. Thus, Kf could invade a neighboring configuration Kq, and Kf could 
invade both Kq and Kq. However, we recall that Kf and Kf cannot invade Kq because of eq. (|i~8|). 
This means that the invasion of cooperation necessarily stops at some point. 

On the other hand, Kq and could invade Kf and Kf because of eq. (0). But this kind of invasion 
does not happen if eq. ( p0| ) is satisfied. This can be explained by looking at Fig. [5] where the payoffs 
in the border region are shown under the assumption that eq. ( p0| ) is valid. In the example shown, 
the payoff of configuration Kq is higher than Kl or Kf because of eq. (^)- But because of eq. (p0|), 
af > ag, thus the invasion of Kq cannot take place, i.e. the border remains at its current position. 
A similar explanation holds for the remaining case Kq. 

In conclusion, one (strong) condition for coexistence is given by eq. (|20"|). For s € {2,3,4} this 
means basically three different inequalities. If all of them are satisfied, the cooperators become the 
global majority, because they are always allowed to invade Kq, Kq, whereas defectors in Kq who 
could possibly invade the cooperation domain are stopped, as explained above. We could relax 
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o 



a\ or a 



Figure 5: Payoffs in the border region between domains of cooperators (Kf) and defectors (Kq) with 
the assumption that eq. (|2^) is valid. It explains that configuration Kq cannot invade neighboring 
configurations K\ or K\ because of the higher payoff of configuration Kf that "backs" them from 
the other side. 

the condition of eq. ( p0| ) that causes this stop. So, let us assume the case that only two of the 
coexistence inequalities are satisfied, i.e. 

a\ > a^ 1 (s € {2, 3}) ; a\ < (21) 

Then, the defector configuration Kq can invade the cooperation configurations Kf, Kf, which 
eventually leads to the domination of defectors, i.e. the cooperators become a global minority. We 
note that in this case we will not find stable coexisting domains, but a nonstationary pattern that 
appears to be spatial chaos, as also shown in the next section. 

If the strong condition for coexistence, eq. ( f20| ) is not satisfied, a weaker condition for coexistence 
could still hold true: 

a^ 1 > al > a S Q- 2 (sG{3,4}) (22) 

This condition would lead to an even greater domination of defectors, because only the configuration 
Kq can be invaded by cooperators, while all other ones are protected from this. In this case, as we 
will also show by means of computer simulations, the cooperators will only survive in very small 
clusters. 

A very special case arises if none of the above coexistence inequalities are satisfied, but instead 
either one of the following conditions holds: 

a l Q<a\< al (23) 
al < a\ ; a\< a\ (24) 
al<a\; a\ < < a\ (25) 

In this case, if we start from the special initial condition of only two domains separated by a 
planar interface, we find that this is also a stable configuration. But any deviation from this initial 
condition will eventually lead to an invasion of defectors. In particular, if we start from a random 
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initial distribution of cooperators and defectors, invasion of defectors will always take place under 
one of the conditions definded by eqs. (f23|)-(|25|). Therefore, this case has been named unstable 
coexistence (d) in Table [fl. It denotes the transition from the case of complete invasion of defectors 
to the case of stable coexistence between cooperators and defectors. 

The only remaining case to be explained is Kq in the border region, which arises if cooperation 
invades into the domain of defection from different sides. Then, it can happen that a single defecting 
agent is trapped within the domain of cooperators. Because of Oq > af, defection will invade all 
neighboring sites during the next generation, this way leading to a configuration Kq in the border 
region. This could lead to a growth of the domain of defectors if the conditions for complete 



invasion, eq. (19) is satisfied (see also Table |IJ). But if these conditions are not satisfied, there are 
various possibilities dependent on whether three, two or one of the inequalities for coexistence are 
satisfied (see also Table ^) . Kq is surrounded by other configurations Kq , Kq or Kq in the larger 
neighborhood. If eq. ( p0| ) is valid and only Kq, Kq are present, then in the next generation they 
will be occupied by cooperators, as explained above, leaving the defector in the central cell in its 
trapped situation, again. So, a cycle of Kq and Kq alternating is created. If Kq is present, this will 
create a stable border between the spatial domains of cooperators, this way leading to coexistence 
as explained above, too. If eq. (|2l| ) is valid and Kq is present, this will lead to the split-up of the 
domains of cooperators. If eq. ( p2| ) is valid and Kq is present, this will lead to a growth of the 
defector domain. 

In the following section, we will apply the analytical conditions derived for the invasion and coex- 
istence of cooperators and defectors to find out critical values of the payoff matrix that may lead 
to the predicted spatial patterns. 



4 Critical payoff values for invasion and coexistence 
4.1 Derivation of a phase diagram 

The investigations of the previous section have provided us with a set of inequalities for the payoffs 
a S Q, in order to find certain patterns of invasion and coexistence of cooperators and defectors. The 
possible payoffs a s e , eq. ([u]) depend on the set of variables {R; S;T; P} of the payoff matrix, eq. 
(Q). In the following, we fix the values for R = 3 and 5 = 0, i.e. the payoffs for a cooperating agent 
i, to the standard values, and this way derive the critical conditions for the payoffs for a defecting 
agent, T and P. With the known payoffs a s e , eq. (|l6[), we find from the different inequalities ( ^0|) 
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the following set of conditions: 



(a) 


a\ 




a 3 => 


3T + P 


= 12 


(b) 


a\ 




a 2 Q => 


2T + 2P 


= 9 


(c) 


2 

a- 1 




a\ => 


T + 3P 


= 6 


(d) 


4 




al => 


2T + 2P 


= 12 


(e) 


4 




a l Q => 


T + 3P 


= 9 


(f) 








T + 3P 


= 12 



(26) 



Additionally, we always have 

3 < T < 6 ; < P < 3 (27) 

because of the ineqs. (||), (@). With these restrictions, we are able to plot a phase diagram in the 
{P, T} parameter space, Fig. || where the conditions ( p6|) mark the different boundaries between 
the phases. The labels A-E in Fig. ^ denote the parameter regions associated with the different 
stationary patterns and shall be explained in detail in the following. 



4.2 Complete invasion of defectors (region E) 

For the complete invasion of defectors into the domain of cooperators, the necessary condition of 
eq. (|l9| ) applies, i.e. 

^>R (28) 
With the fixed values for R and P and eq. (^?j) we find 

12-3P<T<6 if 2<P<3 (29) 

This condition defines region E in the parameter space shown in Fig. |6|. Thus, for the appropriate 
{T, P} values the stationary state is always entirely dominated by the defectors regardless of the 
initial conditions, therefore no further computer simulations are presented here. 

4.3 Coexistence with a majority of cooperators (region A) 



Coexistence between cooperators and defectors becomes possible if eq. ( p0[ ) is fulfilled. As we have 
shown in Sect. ||, this leads to different cases dependent on how many inequalities of eq. ( |20| ) are 
satisfied. If all of them are fulfilled, this leads to the conditions: 

2R + 2S T + 3P 3R + S 2T + 2P „ 3T + P 

'a > j ; : > : ; R > : (30) 
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Figure 6: Phase diagram in the {T, P} parameter space for fixed values of R = 3 and S = of the 
payoff matrix, eq. (||). The different lines result from eq. d26| ) (a — /) and eq. (27). The areas A — E 



(in different gray scales) indicate parameter regions that lead to particular spatial patterns in the 
stationary limit. For a detailed explanation see text. Note that the "classical" parameter values 
{R; S; T; P} = {3; 0; 5; 1} are just on the border between regions C and D. 

With the fixed values for R and S we find: 

T + 3P<6; 2T + 2P<9; 3T + P<12 (31) 

where additionally eq. ( p7| ) applies. The solution is then given by: 

3 < T < 4 - =J if <P < 0.75 , . 

3 < T < 6 - 3P if 0.75 < P < 1.0 

These two conditions define region A of Fig. |(| Eq. (|33|) gives the respective payoffs a^, eq. (|l6|) for 
the choosen set of parameters, {R; S;T; P} = {3.0; 0.0; 3.5; 0.5}. The a s e can be used to verify the 
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general considerations of Sect. H. They are printed here to simplify the comparison. 



4 

3.0 



2.25 



1.5 



0.75 



0.0 



3.5 



2.75 



2.0 



1.25 



0.5 



(33) 

Note that the af are the same as in the "classical" case, eq. (|16|) since they only depend on the 
values of R and S that are not changed. For the appropriate {T, P} values the stationary state 
always shows a coexistence between cooperators and defectors where the cooperators are a majority. 
Note, that this parameter region is just opposite to the region E, Fig. |6| where the stationary state 
is characterized by an extinction of the cooperators. 

The coexistence between cooperators and defectors shall be also demonstrated by means of com- 
puter simulations. To elucidate the dynamics, we have choosen two different initial conditions for 
the simulations, shown in Fig. [7|. The left part of Fig. ^ shows a regular situation of only one cluster 
of cooperators is a sea of defectors, i.e. the cooperators are the absolute minority. For this initial 
condition, Fig. ^ shows the evolution of the global frequency for the different parameter regions A, 
B and C. The right part of Fig. ^ shows a random initial distribution of cooperators and defectors 
with the same initial frequency, /(0) = 0.5. In order to study the influence of the initial frequency 
on the long-term dynamics, we will also choose random distributions with different values of /(0). 





Figure 7: Initial spatial distributions of cooperators (grey) and defectors (black) on a CA of size 
N = 40 x 40, used for the computer simulations: (left) One cluster of 9 cooperators, /(0) = 0.0056, 
(right) random spatial distribution of cooperators and defectors, shown for /(0) = 0.5. 







+ 






(5) 




(10) 




(20) 




(30) 



Figure 8: Time series of the spatial distribution of cooperators (grey) and defectors (black) on 
a CA of size N = 40 x 40. The time is given by the numbers of generations in brackets. Initial 
condition: One cluster of 9 cooperators, Fig. |/](left). Parameters for the payoff matrix, eq. (Q): 
{R; S; T; P} = {3.0; 0.0; 3.5; 0.5} (region A). 
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Figure 9: Global frequency of cooperators, /, eq. (|T|), vs. time (number of generations). Initial 
condition: One cluster of 9 cooperators, Fig. ^(left), /(0) = 0.0056. Parameters for the payoff 
matrix, eq. (|), {R; S;T; P}: (A) {3.0; 0.0; 3.5; 0.5} (region A), (B) {3.0; 0.0; 3.9; 0.5} (region B), 
(C) {3.0; 0.0; 4.5; 1.0} (region C). 




Figure 10: Final (steady state) spatial distribution of cooperators (grey) and defectors (black) 
on a CA of size A = 40 x 40. Initial condition: random spatial distribution of cooperators and 
defectors, left: /(0) = 0.5, middle: /(0) = 0.75, right: /(0) = 0.9. Parameters: {R;S;T;P} = 
{3.0; 0.0; 3.5; 0.5} (region A). 



From the computer simulations shown in Fig. || and Figs. |8|, [Tl], we can draw the following conclu- 
sions in agreement with the general discussion in Sect. || and the discussion of Fig. ^ in Sect. 2.2: 
We observe the formation of spatial domains of cooperators that are separated by narrow regions 
of defectors. Only the special case of a singular initial cluster shown in Fig. || leads to a regular 
stationary final pattern where cooperators and defectors are clearly separated into two domains. In 
the stationary state we find a stable coexistence between cooperators and defectors. The cooperators 
become the majority (/ > 0.5) almost independent of their initial spatial configuration and initial 
frequency as shown in Fig. |^A and Fig. [ll]. The stationary frequency is almost constant (with very 
small periodic changes). 
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Figure 11: Global frequency of cooperators, /, eq. (||), vs. time (number of generations). The 
data have been obtained from simulations with three different initial frequencies: (a) /(0) = 0.5, 
(b) /(0) = 0.75, (c) /(0) = 0.9. See also Fig. 10 for the final spatial distributions. Parameters: 
{R; S; T; P} = {3.0; 0.0; 3.5; 0.5} (region A). 



4.4 Coexistence with a minority of cooperators - spatial chaos (region B) 



If only two of the coexistence conditions are satisfied, i.e. eq. ( |2lD applies, this leads to: 
2R + 2S T + 3P 3R 2T + 2P „ 3T + P 

A > A 5 ~~T > A > R < A ( 34 ) 

4 4 4 4 4 

With the fixed values for R and S, we find: 

T + 3P < 6 ; 2T + 2P < 9 ; 3T + P > 12 (35) 

With eq. (^7j), this eventually results in the solution: 

4-^<T<4.5-P if 0<P < 0.75 (36) 

This condition defines region B of Fig. ^. Eq. (|37|) gives the respective payoffs a s d , eq. (|l6|) for 
the choosen set of parameters, {R; S;T; P} = {3.0; 0.0; 3.9; 0.5}. The a s e can be used to verify the 
general considerations of Sect. H. 



4 

3.0 



2.25 



1.5 



0.75 



0.0 



3.9 



3.05 



2.2 



1.35 



0.5 



(37) 

For the appropriate {T, P} values, there is always a nonstationary coexistence between cooperators 
and defectors where the cooperators are a minority. This is also shown for the two different initial 
conditions, Fig. ^ in the computer simulations of Fig. 12, Fig. Fig. |^B and Fig. [l4|. 
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Figure 12: Time series of the spatial distribution of cooperators (grey) and defectors (black) on 
a CA of size N = 40 x 40. The time is given by the numbers of generations in brackets. Initial 
condition: One cluster of 9 cooperators, Fig. ^(left). Parameters for the payoff matrix, eq. (^|): 
{R; S; T; P} = {3.0; 0.0; 3.9; 0.5} (region B). 




Figure 13: Spatial distribution of cooperators (grey) and defectors (black) on a CA of size N = 
40 x 40 after G = 100 generations. Note, that there is no stationary pattern asymptotically. Initial 
condition: random spatial distribution of cooperators and defectors, left: /(0) = 0.5, middle: /(0) = 
0.75, right: /(0) = 0.9. Parameters: {R; S; T; P} = {3.0; 0.0; 3.9; 0.5} (region B). 



The time series of Fig. 12 - that should be compared to Fig. || for parameter region A - shows that 
the initial cluster of cooperators first grows, but then splits up into smaller clusters, which grow, 
split up again and may dissappear. This leads to a non-stationary spatial distribution even in the 
long run. In fact, it has been already argued by Nowak (3^] that this regime can be characterized 
as spatiotemporal chaos. If we look for the time dependence of the global frequency in this case, 
Fig. |9|B and Fig. 14, we find no convergence to a (quasi)stationary value as in Fig. 11, but rather 
large variations over time. Noteworthy, in the average the global frequency of cooperators is below 
0.5, i.e. they are indeed the minority almost independent of the initial frequencies. Further, we note 
that the cooperators still form large clusters, as shown Fig. 13 that can be compared to Fig. [n] for 
region A. 



4.5 Coexistence with cooperators in small clusters (region C) 



Eventually, we also show examples for the weaker coexistence condition, eq. (\22\j. In this case, the 
conditions are given as: 

2T + 2P 3R + S T + 3P 3T + P n 2T + 2P 

a > a > a ' 'a > R> : (38) 
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Figure 14: Global frequency of cooperators, /, eq. (||), vs. time (number of generations). The 
data have been obtained from simulations with three different initial frequencies: (a) /(0) = 0.5, 
(b) /(0) = 0.75, (c) /(0) = 0.9. See also Fig. 13 for the final spatial distributions. Parameters: 
{R; S; T; P} = {3.0; 0.0; 3.9; 0.5} (region B). 



With R = 3 and S = 0, this results in: 

T + 3P < 9 ; 2T + 3P < 12 ; 2T + 2P > 9 ; 3T + P > 12 
where additionally eq. ( |27| ) applies. The solution of eq. (39) is then given by: 



4 - P/3 < T < 9-3P if 
4 - P/3 < T < 6 - P if 
4.5 - P < T < 6-P if 



1.5 < P < 1.875 
0.75 < P < 1.5 
0.0 < P < 0.75 



(39) 



(40) 



These three conditions define region C of Fig. [ 
for the choosen set of parameters, {R; S;T; P} 
the general considerations of Sect. ||. 



j. Eq. (|4l| ) gives the respective payoffs a s d , eq. ( |l6|) 
= {3.0; 0.0; 4.5; 1.0}. The a s can be used to verify 
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(41) 

For the appropriate {T, P} values, the stationary states are characterized by a stable coexistence 
between cooperators and defectors where the cooperators survive only in small clusters, while the 
defectors dominate. This is also shown for the two different initial conditions of Fig. [j] in the 
computer simulation of Fig. Ilf 



Fig. 15 



Fig. |IJ, Fig. |C and Fig. [17 
that should be compared to Fig. 



for parameter region A and Fig. 12 for region B 



indicates that the initial cluster grows only very little in size, i.e. from 9 to 13 cooperators, and then 
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Figure 15: Time series of the spatial distribution of cooperators (grey) and defectors (black) on 
a CA of size N = 40 x 40. The time is given by the numbers of generations in brackets. Initial 
condition: One cluster of 9 cooperators, Fig. ^(left). Parameters for the payoff matrix, eq. (^|): 
{R; S; T; P} = {3.0; 0.0; 4.5; 1.0} (region C). 



Figure 16: Final (steady state) spatial distribution of cooperators (grey) and defectors (black) 
on a CA of size N = 40 x 40. Initial condition: random spatial distribution of cooperators and 
defectors, left: /(0) = 0.5, middle: /(0) = 0.75, right: /(0) = 0.9. Parameters: {R;S;T;P} = 
{3.0; 0.0; 4.5; 1.0} (region C). 

remains stable at its borders. However, if the initial configuration would have more than one cluster, 
then not all of them may survive. If two cooperating clusters are very near, then the configuration 
Kq may appear between them, which leads to an invasion of the defectors in the next step and the 
further discussion of Sect. |3.2j applies. Thus, several small clusters may only survive if there is a 



certain distance between them. This can be also seen in Fig. 16 that shows the stationary spatial 



distributions for parameter region C, in comparison to Fig. 10 for region A and Fig. ^ for region 
B. We observe only very small clusters at a certain distance, the number of which further depend 
on the initial frequency of cooperators. The evolution of the global frequency, Fig. ||C and Fig. [l?], 
also clearly shows that the cooperators survive only as a small minority, which should be compared 
to Fig. and Fig. ||. 

4.6 Special case of unstable coexistence (region D) 

The last case to be discussed is the special case of eq. (|2^ ) - (]25|) , which leads to coexistence only for 



very special initial conditions, as discussed in Sect. 3^. From eq. (|23|), we find: 



T + 3P 2T + 2P 

— T~ < R < 7 (42) 
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Figure 17: Global frequency of cooperators, /, eq. (||), vs. time (number of generations). The 
data have been obtained from simulations with three different initial frequencies: (a) /(0) = 0.5, 



(b) /(0) = 0.75, (c) /(0) = 0.9. See also Fig. 16 for the final spatial distributions. Parameters: 
{R; S; T; P} = {3.0; 0.0; 4.5; 1.0} (region C). 

With R = 3, S = 0, we have following conditions: 

T < 12 - 3P ; T < 6 - P (43) 
where additionally eq. (]27|) applies. This leads to the solution: 

6 - P < T < 6 if < P < 3 (44) 
Similarly, the special case of eq. (p4j) with R = 3, S = and eq. (27) leads to: 



2T + 2P „ 3R + S T + 3P 

2T + 2P < 12 ; 9 < T + 3P (45) 

with the solution: 

3<T<6-P if 2<P<3 (46) 
while the special case of eq. ( p5| ) results in: 

3T + P „ 2P + 25 T + 3P 3R + S 

a < R 5 a < a < a 

4 4 4 4 

3T + P<12 ; 6<T + 3P<9 (47) 



with the solution: 



3 < T < 12 j P if 1.0 < P < 2.0 
6 - 3P < T < 12 j P if 0.75 < P < 1.0 



(48) 
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The three solutions fl44]), (fig), together define parameter region D of Fig. [6|. Provided the 
appropriate {T, P} values, we observe in this case only an unstable coexistence of cooperators and 
defectors, i.e. for an random initial distribution the dynamics will always lead to an invasion of 
the defectors until a complete extinction of the cooperators - rather similar to the dynamics in the 
adjacent parameter region E. Only for the special case of two domains with a straight border, the 
coexistence remains stable. 

Looking at the phase diagram of Fig. ^, we notice that region D separates the regions of stable 
coexistence (A-C) from the region of complete invasion of defectors (E), i.e. D denotes the transition 
region between the two different stationary regimes, and the border between the two regions C and 
D just marks the transition from stability (i.e. stable coexistence) to instability (i.e. unstable 
coexistence). We further notice that the "classical" parameter values {R; S; T; P} = {3; 0; 5; 1} are 
just on the border between regions C and D, i.e. for a random initial distribution the classical 
spatial 5-person game will always lead to the complete invasion of defectors and the extinction of 
cooperators, as already observed by means of computer simulations. 



4.7 Dynamics at the border regions 



So far, we have discussed the dynamics for the pure parameter regions, A-E. However, the question 
of interest is how the system behaves if we choose the parameters of the payoff matrix on the border 
between two such regions. We have already mentioned the case of the C|D border, that marks the 



transition from (stable) coexistence to invasion. Fig. 18 presents results of computer simulations 
for the A IB and B|C border. 
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Figure 18: Global frequency of cooperators, /, eq. ((!]), vs. time (number of generations). The data 
have been obtained from simulations with three different initial frequencies: (a) /(0) = 0.5, (b) 
/(0) = 0.75, (c) /(0) = 0.9. Parameters for {R; S;T; P}: (left) {3.0; 0.0; 3.833; 0.5} (border between 
regions A and B), (right) {3.0; 0.0; 4.0; 0.5} (border between regions B and C). 
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As we see from the time evolution of the global frequencies, the dynamics in this case is basically 
the same as for region B, i.e. there is no (quasi)stationary regime as for regions A or C, and we 
may expect a spatiotemporal chaos again. Despite this, the average global frequency in the long 
run is also decreasing when going from region A to C. In Fig. |i~8|(left), i.e. on the border A|B it is 
below the values of region A, where the cooperators appear as a majority, Fig. |ll], but above the 
values of region B, where they already appear as a minority, Fig. [l4|. But in Fig. |i~8|(right), i.e. on 
the border B|C it is below the values of region B, but still above the values of region B, where the 
cooperators appear as a clear majority, Fig. [L7]. 

4.8 Nowak's results revisited 

For the parameter region B we already mentioned the existence of spatiotemporal chaos in the 
distribution of cooperators and defectors. This has been first observed by Nowak |}2| by means of 
computer simulations. Based on our detailed theoretical investigations above, we are now able to 
derive the critical parameter values for Nowak's simulations. 

Different from the standard values of the prisoner's dilemma, {R; S; T; P} = {3; 0; 5; 1}, Nowak has 
used the following values for the payoff matrix: 

{R;S;T;P} = {l;0;b;0} (49) 

i.e. all payoffs are fixed, while only the payoff for the defecting agent playing with a cooperating 
agent is an adjustable value, b. In order to fulfill the conditions for the payoff, eqs. (||), (Q), b can 
have only values of 1 < b < 2. Further, we notice that the payoff matrix given by eq. (|49|) does not 
strictly fulfill the conditions of the prisoner's dilemma, because of P = S in this case. 

We can now apply the conditions derived for the different regions to the payoff matrix of eq. (ff9|). 
The results are given in Table 0. 



Table 3: Parameter range of the payoff b, eq. ([49J) to obtain the different spatial patterns in the 
asymtotic state. 



stationary state 


condition 


range of b values 


A: coexistence with large domains of cooperators 


eq. © 


1.0 < b < 1.33 


B: coexistence with spatial chaos 


eq. © 


1.33 < b < 1.5 


C: coexistence with small clusters of cooperators 


eq. © 


1.5 < b < 2.0 



We note again that these regions have been found in [32] by means of computer simulations, while we 



have confirmed the results based on an analytical investigation of the stability conditions. Further, 
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we want to point out that the other two dynamical regimes, i.e. E: complete invasion of defectors 
and D: unstable coexistence, do not exist for the payoff matrix, eq. (49). 



5 Interactions in a larger neighborhood 

In this section, we want to implement the "exact" (or complete) 5-person game for the one-shot 
PD game on the square lattice - which is to our knowledge not investigated so far. Since the payoff 
matrix for the 5-person game has been computed in this paper using the concept of the 2-person 



game, Sect. 2.2, we will continue to explain the dynamics of the 5-person game in terms of 2-person 



games. So far, we have investigated the spatial 5-person game under the assumptions given in Sect. 



2.2 , namely (i) decomposition into independent, simultaneous 2-person games, (ii) each agent only 
plays with its m nearest neighbors. This has reduced the game to m 2-person games played by each 
agent. 

In the following, we want to discuss a different variant of the game which instead of (ii) assumes 
that in a neighborhood of n = 5 each agent plays a 2-person game with every other agent in 
this neighborhood. For a given neighborhood, this increases the number of 2-person games to 
m x n/2 = 10. We further have to take into account that agent % itself is additionally part of 
m different neighborhoods of size n = 5 centered around its nearest neighbors at the positions 
ij (j = l,...,m). This results in the consideration of a larger neighborhood of size n = 13 that 
includes also the r second nearest neighbors of agent i (see Fig. |l|). The total number of 2-person 
games played independently in this larger neighborhood is given by mn 2 /2 = 50, but we can easily 
verify that agent i at position j = participates only in 20 of these. Specifically, he plays 4 games 
in his "own neighborhood" (j = 0) and 4 times 4 games in the neighborhoods of his neighbors 
(j = 1, ...,m). In these 20 games, he meets always twice (in two different 5-person games) both 
with his m nearest neighbors and with the neighboring agents at the positions j = 6, 8, 10, 12, but 
only once with the agents at the second nearest neighbor positions j = 5, 7, 9, 11 (cf. Fig. [I]). 

This in turn results in a different payoff, dependent on the number of cooperators and defectors 
in the larger neighborhood of agent i. Since the payoff is again calculated from the independent 
2-person games, we can still use eq. (|TT|) for a,i, but we have to recalculate the fractions z\ and zf 




according to the modified game. I.e. eq. ( jlOj ) has to be replaced by: 

= 1-4 (0 6 {0,1}) (50) 

Eq. (|50j) can be interpreted in a way that agent i - in addition to his nearest neighbors - now 
also interacts with his second nearest neighbors, but if they are not adjacent to his place (i.e. 
j £ 5,7,9,11), this interaction occurs less frequently - or, the payoff is counted with a smaller 
weight, respectively. 
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We do not intend to give here a detailed analysis of this modified case, the effect of the weighted 
interaction in the larger neighborhood shall be rather demonstrated by some computer simulations 
that allow a comparison to the previous results. For the dynamics of the modified game the update 



rule, eqs. fll3|), fll4| ) still applies, i.e. agent i adopts the C or D behavior from the agent that received 
the highest payoff in its neighborhood of n = 5. The parameters of the payoff matrix, eq. (||) have 
been choosen in region A of the phase diagram, Fig. |6|, where we found the spatial coexistence 
defectors (as the minority) and cooperators (as the majority), the payoff values a s 9 are given by eq. 

For the computer simulations, we have again used the initial spatial configurations shown in Fig. 
0, i.e. either one small cluster of cooperators, or a random intial distribution of cooperators and 
defectors with different intial frequencies. Fig. |l9]- that shall be compared to Fig. || - shows that in 
the modified case the domain of cooperators considerably increases, i.e. it reaches / = 0.95 compared 
to / = 0.52. Further the dynamics occurs faster (compare the snapshot after 30 generations in Fig. 
H with the one after 20 generations in Fig. [i~9| ) and the border between the two domains remains 
planar during the evolution. 




♦ 



1(10) (20) 



1(30) 



Figure 19: Time series of the spatial distribution of cooperators (grey) and defectors (black) for 
the modified game, eq. (]50D . The time is given by the numbers of generations in brackets. Initial 
condition: One cluster of 9 cooperators, Fig. 0(left). Parameters for the payoff matrix, eq. @): 
{R; S; T; P} = {3.0; 0.0; 3.5; 0.5} (region A). 




Figure 20: Time series of the spatial distribution of cooperators (grey) and defectors (black) for 
the modified game, eq. (|50|). The time is given by the numbers of generations in brackets. Initial 
condition: /(0) = 0.5, random spatial distribution of cooperators and defectors, Fig. fright). 
Parameters: {R; S;T; P} = {3.0; 0.0; 3.5; 0.5} (region A). We note that the further evolution leads 
to a final spatial distribution similar to the one shown in Fig. |22](left). 



The influence of the larger neighborhood on the evolution of the spatial patterns becomes more 
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Figure 21: Global frequency of cooperators, /, eq. ((!]), vs. time (number of generations) for the 
modified game, eq. (|50|). The data have been obtained from simulations with three different initial 
frequencies: (a) /(0) = 0.5, (b) /(0) = 0.75, (c) /(0) = 0.9. See also Fig. || for the final spatial 
distributions. Parameters: {R; S;T; P} = {3.0; 0.0; 3.5; 0.5} (region A). 




Figure 22: Final (steady state) spatial distribution of cooperators (grey) and defectors (black) 
for the modified game, eq. (|50|), Initial condition: random spatial distribution of cooperators and 
defectors, left: /(0) = 0.5, middle: /(0) = 0.75, right: /(0) = 0.9. Parameters: {R;S;T;P} = 
{3.0; 0.0; 3.5; 0.5} (region A). 

visible when we start the simulations from a random initial distribution, as shown in Fig. The 
corresponding evolution of the global frequency f(G) is shown in Fig. [H](a). These figures should 
be compared to Fig. || and Fig. ||[ Starting with an initial frequency /(0) = 0.5, we find that the 
cooperators during the first three generations almost cease to exist, they survive only in a few 
rather small clusters. But it is worth to be noticed that this situation then changes drastically: 
the small clusters grow into a few large domains that are separated by only tiny borders formed 
by defectors. Thus, compared to the previous simulations, we now find (i) less separated domains, 
and (ii) a much greater domination of the cooperators in the final state, i.e. / = 0.95 in this case. 
Interestingly, this domination becomes the greater, the less the inital frequency is (cf. Fig. |2l| ) - 
but a the same time also the risk increases that cooperators die out during the first generations. // 



28/1 



Frank Schweitzer, Laxmidhar Behera, Heinz Miihlenbein: 
Evolution of Cooperation in a Spatial Prisoner's Dilemma 
Advances in Complex Systems, vol. 5, no. 2-3, pp. 269-299 



they survive, their "comeback" is overwhelming. 

Fig. [2^ shows the spatial final distributions that correspond to the different frequencies in Fig. 
21. We notice that the different initial frequencies /(0) do not only have a strong impact on final 
frequencies, but also on the number of domains formed during the evolution. The higher the initial 
frequency of cooperators, the less is the final frequency of cooperators and the more they are 
splitted into separated domains. This effect was not so pronounced for the simulations shown in 



Fig. 1C - thus, we may conclude that the modified game, i.e. the consideration of the spatially 



heterogeneous interaction in the larger neighborhood may lead to nontrivial diversification in the 
spatial dynamics, which will be investigated in a forthcoming paper. 



6 Conclusion 



In this paper we have investigated the spatial organization of cooperating and defecting agents 
distributed on a square lattice. From his interaction with other agents, each agent receives a payoff 
that can be compared to the payoffs of the other agents. Based on this outcome the agent can in 
the next generation adopt the more successful behaviour, either to cooperate (C) or to defect (D). 

Different from a mean-field approach that assumes a panmictic population where each agent in- 
teracts with every other agent, we have considered a spatially restricted case, where each agent 
interacts only with his neighbors. This results in a spatial 5-person game that has been discussed 
in two variants: (i) each agent interacts only with his four nearest neighbors, (ii) each agent in the 
neighborhood of 5 interacts with any other agent in this neighborhood. Such a distinction leads to 
a different number of 2-person games played simultaneously, but independently by each agent in a 
given neighborhood, i.e. 4 games in the first case and 10 games in the second case. 

The main part of the paper is devoted to the first case. Based on the exact calculation of the payoffs 
for the possible encounters in a given neighborhood, we have derived analytical expressions (in terms 
of inequalities) to characterize the different spatial organizations of cooperators and defectors. The 
results are concluded in a phase diagram for the {T, P} parameter space of the possible payoffs for 
defectors, Fig. ||. We could identify five different dynamical regimes (A-E), each characterized by a 
distinct spatiotemporal dynamics and a corresponding final spatial distribution. We found that for 
arbitrary initial conditions parameters choosen from regions E and D will always lead to a spatial 
invasion of the defectors and eventually results in a complete extinction of the cooperators. This 
steady state agrees with the steady state of the mean-field dynamics, where the invasion of defectors 
is the only possible outcome. I.e., if the values of the payoff matrix are choosen from regions D 
and E, then space plays no longer a role in the determination of the steady state, that is the same 
regardless of whether the interaction occurs only between nearest neighbors or between all agents 
in the system. 
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In addition to that we could also identify parameter regions (A, B, C) characterized by a spatial 
coexistence between cooperators and defectors. A detailed analysis could reveal the conditions under 
which the cooperators could survive either as a majority organized in large spatial domains, or as 
a minority organized in small non-stationary domains or in small clusters. 

The analytical results obtained have been further applied to a spatial game introduced by Nowak 



and May [32]. The parameter findings obtained there by means of computer simulations could be 
confirmed by our analytical approach. 

In the last part of the paper, we have focussed on the variant (ii) of the 5-person game, where 
each agent interacts with any other agent in his neighborhood. If this "true" 5-person game is 
put on a rectangular lattice - as we did here to our knowledge for the first time - it results 
in interesting effects. Since each agent is part of different spatial 5-person games (played in his 
immediate neighborhood) the modification eventually leads to the consideration of the second- 
nearest neighbors. We could show that the interaction in this larger neighborhood can be described 
as a non-uniform spatial game, where each agent plays 2-person games more frequently with his 
adjacent neighbors than with his second-nearest neighbors. As the result, we found by means of 
computer simulations that during the first stages of the evolution the risk considerably increases 
that the cooperators ceases to exist, but if they survive, they can become a much larger majority 
than in the "simplified" case (i). In turn, it is the defectors that only survive in thin borders. This 
is very interesting since true 5-person games are rather complex, and it is known from mean-field 
investigations that achieving cooperation becomes even more difficult in multi-person games. Thus, 
the analytical investigations in this paper shall be also applied to the true spatial 5-person game, 
in order to reveal its critical conditions. 

We can conclude that space indeed plays a definite role in the evolution of cooperation, because 
a spatially restricted interaction may lead to a global cooperation, even if individual rationality 
tilts toward defection. Even in a one-shot PD game it is possible to find a majority of cooperators, 
provided a locally restricted interaction is considered. 
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